Vision based scene understanding has been used in various systems. Based on such an understanding of the scene, a variety of applications can be derived. One such application is vision based obstacle detection that can help in enabling robotic vision based systems and autonomous ground vehicles. Perception of obstacles and avoidance of obstacles is particularly important in autonomous vehicles. Some typical autonomous vehicle/navigation systems include applications related to perimeter following, navigation in unknown terrains, autonomous vehicles on the road, navigation in cluttered environments etc. These situations often include challenging and complex scenarios requiring hybrid sensor usage such as laser, GPS and wireless sensors. Vision sensor (camera) systems are used to supplement other sensors in these situations.